Search Results: "Steve Langasek"

8 February 2014

Sam Hartman: Debian: Init Interfaces Our Users *and* Free Software

Recently, I ve been watching the Debian Schadenfreude related to init systems. For those who do not know, Debian is trying to decide whether Systemd or Upstart will be used to start software on Debian. There are two other options, although I think Systemd and Upstart are the main contenders. Currently the Technical Committee is considering the issue, although there s some very real chance that the entire project will get dragged through this swamp with the general resolution process. This is one of those discussions that proves that Debian truly is a community: if we didn t have something really strong holding us together we d never put up with something this painful. Between the accusations that our users now know the persecution European pagans faced at the hands of Christians as the Systemd priests drive all before them, a heated discussion of how the Debian voting system interacts with this issue, two failed calls for votes, a discussion of conflicts of interests, and a last-minute discussion of whether the matter had been appropriately brought before the Technical committee (and if so, under what authority), there s certainly schadenfreude to go around if you re into that sort of thing. However, through all this, the Technical Committee has been doing great work understanding the issues; it has been a pleasure to watch their hard work from the side lines. I d like to focus on one key question they ve found: how tightly can other software depend on the init system. Each init system offers some nice features. Upstart has an event triggering model. Systemd manages login sessions and at least in my opinion has a really nice service file format. Can I take advantage of these in my software. If I do, then users of my software might need to use a particular init system. Ian Jackson argues that we should give our users the choice of what init system to run. He reminds us that Debian is a community that supports diversity and diverse goals. We support multiple desktop systems, web browsers, etc. Wherever we can, we support people working on their goals and give developers the opportunity to make it work. When developers succeed, we give our users the opportunity to choose from among the options that have been developed. I think of Ian s argument as an appeal to part of our Social Contract. Clause 4 of the social contract begins:
Our priorities are our users and free software.
We will be guided by the needs of our users and the free software community. We will place their interests first in our priorities. We will support the needs of our users for operation in many different kinds of computing environments.
Ian is right that our users will be served by choice in init systems. However, this trade off needs to be balanced against the needs of the free software community. Diversity is an important goal but it should not come at the price of stagnation and innovation. If I want to avoid using init scripts because they don t provide restart on failure, because they are hard to write correctly, and because they don t provide access to modern security isolation techniques, I should be able to do that. If Systemd service files provide a superior solution, I should be able to work toward using them. If the desire for init system diversity shuts down my ability to find like-minded people who want to take advantage of improvements in init systems and work towards that goal, then we ve significantly damaged Debian as a forum for technical cooperation. Early in my work as a Debian Developer, Anthony Towns taught me an interesting principle. Those who do the work get significant influence over how and what gets done. Ian s right that we need to enable people to work towards choice in init systems. Those like-minded people need to be given a chance to find each other and pursue their goal. However the cost of success should be on the shoulders of those who value choice in init systems. It should not come at the cost of preventing people from depending on improvements in init systems. The best proposal I ve seen for balancing this is to enumerate stable interfaces that people want to use. Things like the logind interface or my favorite the service file interface. It needs to be possible to make another implementation of the interface. The interface needs to be stable enough that a dedicated team could have a chance of catching up with an implementation. At some point during the release cycle such interfaces would need to be frozen. However, I don t think it is reasonable to mandate that there are multiple implementations of the interface, only that there could be. The point is to give people a chance to work towards diversity in init systems, not to force it down peoples throats kicking and screaming until they leave the project or ignore our processes. Steve Langasek and Colin Watson seem to be working towards this. It s possible there s another approach besides interfaces. My main concern is the same as Ian s: maintain Debian as a forum for people to pursue their goals and work together. I suspect we see the conflict in these goals differently. I hope that we as a project can explore this conflict and figure out where we have common ground. I commit to exploring how we can work towards init choice in my frameworks; I ask those who prioritize init choice to commit to explore how we can take advantage of new features in their framework.

27 January 2014

DebConf team: Call for sponsorship for DebConf14 (Posted by Steve Langasek)

With DebConf13 behind us, the team is looking forward to DebConf14 this August in Portland, Oregon, USA. As a community-oriented conference, DebConf relies on its network of generous sponsors for funding. Please help us make DebConf14 a success - Become a sponsor today!

19 November 2013

Raphaël Hertzog: Will Debian s technical committee coopt Keith Packard or Philipp Kern?

The process has been ongoing for more than a year but the Debian technical committee is about to select a candidate to recommend for its vacant seat. The Debian Project Leader will then (likely) appoint him (looks like it won t be a women). According to recent discussions on debian-ctte@lists.debian.org, it seems that either Keith Packard or Philipp Kern will join the committee. If you look at the current membership of the committee, you will see: That s very Anglo-Saxon centric (6 out of 7 members). While I trust the current members and while I know that they are open-minded people, it still bothers me to see this important body with so few diversity. Coming back to the choice at hand, Keith Packard is American and Philipp Kern is German. No new country in the mix. I can only hope that Philipp will be picked to bring some more balance in the body.

9 comments Liked this article? Click here. My blog is Flattr-enabled.

31 August 2013

Christian Perrier: DebConf 13: hacking outcome

I *did* some hacking at DebConf, as usual. For once, I had no TODO list: after all, I never complete these, so why make one? Still, I participated to discussions around samba packaging, mostly animated by Ivo De Decker and I think we're making good progress towards samba 4.x packages. The road is long, quite complicated, but we now have a stronger team, with a very active Ivo, Jeroen Dekkers who officially joined, Steve Langasek who still cherishes one of his pet packages (and even branched in our git with the Ubuntu packages). Great work and thanks to Ivo for pushing this forward. I also worked quite actively on the migration of fonts packages to git (I now reached the point where I'm more comfortable with git than SVN, yes, everything can happen). These packages were modernized at the same time and checked for new upstream versions (I have to say that few of these fonts had new upstream versions, indeed). We unfortunately found no time to have a good font BoF in Vaumarcus, indeed...but I'm not sure we would have many things to say. The work is done and done well, in this team. Some progress was made, also, for restoring a working "monolithic" build of D-I. This build gathers together all udebs from unstable, which means it offers a D-I image that uses ALL udebs from nustable.....which is not the default of other images. This would be very helpful for translators who want to check their work as soon as possible. In the future, with a Jenkins task that would build each package at each commit and then build a monolithic image, we could have a way to provide a tes snapshot of D-I git repos.....which could help catching more bugs (or more of my stupid mistakes). So, in general, I consider this a quite successful DebConf when it comes at "real" production.

3 August 2013

Steve Langasek: Network services and Upstart

So Debian bug #710747 led to an interesting discussion the other night on IRC which made me realize there are a lot of people who have yet to understand why sysvinit needs replacing. I can't speak to whether upstart solves this bug in particular. The tftpd-hpa package in Ubuntu (and in Debian experimental) does have an upstart job, and I have used tftpd-hpa on systems whose network interfaces are managed entirely with NetworkManager, and I have not seen this bug; but I can't say that this is more than coincidence. However, I can speak to how upstart provides facilities that address the general problem of starting services that require a working network connection to be present before they start up. Traditionally on Linux systems, init scripts could assume that the network connection is up before starting any services for the target runlevel. With LSB dependencies, you could also express this as
Required-Start:    $network
The trouble with both of these approaches is that they assume that there is a single point in time, early in the boot of the system, when the "network" is up. This was an ok assumption 15 years ago, when the Linux kernel initialized all devices serially and all our network configuration was static. But nowadays, you simply cannot rely on systems booting in such a way that the network is fully up shortly after boot - or in a way that you can block the system boot waiting for the network to be up. If all of our services would cope gracefully with being started without the network, this would be a non-issue. Unfortunately, not all network services are written in such a way as to work without a network; nor do they all cope with dynamic changes to interfaces or addresses. While it would be better if these services were written more robustly, since there's no shortage of daemons written this way, it's convenient that upstart provides tools for coping with this behavior in the meantime. Here's a real (simplified) example of an upstart job for a service that needs to wait for a non-loopback interface before it starts:
start on (local-filesystems and net-device-up IFACE!=lo)
stop on runlevel [!2345]
expect fork
respawn
exec nmbd -D
Now, you might also have a service that you only want to start up when a particular network device comes up. This is easily expressed in upstart, and might look like this hypothetical job:
start on net-device-up IFACE=wlan0
stop on net-device-down IFACE=wlan0
expect daemon
respawn
exec mydaemon
If you need to restart a daemon whenever the set of active network connections changes, that's less straightforward. Upstart doesn't have the notion of a 'restart' rule, so you would need two jobs to handle this - one for the daemon itself that starts on the first network connection, and a second job to trigger a restart of the daemon when the network status changes. For the tftpd-hpa case in the abovementioned bug, this might look like:
$ cat /etc/init/tftpd-hpa.conf
start on runlevel [2345]
stop on runlevel [!2345]
expect fork
respawn
script
        if [ -f $ DEFAULTS  ]; then
                . $ DEFAULTS 
        fi
        exec /usr/sbin/in.tftpd --listen  --user $ TFTP_USERNAME  --address $ TFTP_ADDRESS  $ TFTP_OPTIONS  $ TFTP_DIRECTORY 
end script
$ cat /etc/init/tftpd-hpa-restart.conf
start on net-device-up
task
instance $IFACE
script
        status tftpd-hpa   grep -q start/   stop
        restart tftpd-hpa
end script
For this case, upstart doesn't provide quite so great an advantage. It's nice to be able to use upstart natively for both pieces, but you can do the same thing with an init script plus an if-up.d script for ifupdown, which is what maintainers do today. I think adding a restart on stanza to upstart in the future to address this would be a good idea. Though in any event, this is far simpler to do with upstart than with any if-up.d script I've ever seen for managing an initscript-based service. Between the more friendly declarative syntax, and the enhanced semantics for expressing when to run (or restart) a job, upstart offers clear advantages over other init systems.

2 August 2013

Raphaël Hertzog: My Free Software Activities in July 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (167.67 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook After the successful crowdfunding campaign, I had a bunch of rewards to ship: I subcontracted most of the job but I had to take care of the books with dedication. I also dealt regularly with books/stickers coming back to the sender (due to invalid address or people not picking up their parcels in the post-office). After the rewards, we had to take care to actually finalize the liberation of the French translation. I merged the translations we had in Git and Roland updated/translated a few strings that weren t in the original book in French. Then I have put the book online. Last but not least, I started to work on updating the English book for Debian 7 (Roland started way before me) and we have put some updated chapters up for review. Debian France Elections. After Debian France s general assembly, the new board of administrators voted the officers: I have been re-elected as President, Sylvestre continues as Treasurer but we have a new Secretary in the person of Alexandre Delano . Welcome Alexandre! I did the administrative work to register the new board/officers in the Tribunal d instance and to give access to the internal git repositories to the new members. Galette. I also did a bunch of tests on Galette s new features that Debian France ordered to the upstream author. They should all land in the next upstream release due in the next weeks. \o/ Accounting. I worked on the accounting to bring it up-to-date so that Sylvestre can pick up the work from now on. We re learning how to best use ledger for our needs. PTS rewrite I continued to spend about 12 hours a week to mentor Marko Lalic who is rewriting the Package Tracking System. I m pretty happy with the results so far so I marked him as pass for the mid-term evaluation required by Google. You can have a look at the documentation and the web interface is starting to show some content. The email interface is fully working and I have configured the real PTS to forward all mails to our test instance (pts.debian.net) so that you can use the rewritten PTS for real-life work. Mail your subscription commands to control@pts.debian.net and start using it! Thanks to the test driven development methodology we re using, we re pretty confident that it works reasonably well! :-) I also packaged python-django-jsonfield (still in NEW) since Marko has been using this python module in his code, and filed bug #717900 on sqlite3 to raise a limit that we have hit with queries made by the PTS. Kali Linux I used the Calxeda Highbank node donated to Debian by Offensive Security to test the new -armmp kernel flavor on it. It seemed to work except for a missing network driver (filed in #717269). Misc Debian work Issues with social networks. With the move of identi.ca to pump.io, we don t have any possibility to auto-post status updates based on RSS feeds. Identi.ca s @debian account was also configured to push updates to the @debian account on twitter.com (and from there it was grabbed in the Debian page on Facebook). This is also gone so to limit the damage, I setup twitterfeed.com so that the twitter/facebook accounts continue to have updates). If you re looking for a development project, here s an area that is not well covered by free software! We need code to do what twitterfeed does and we need that code to also support pump.io. Dpkg work. It s been a long time since I last pushed some code to dpkg s git repository. I took care of reworking and merging a patch submitted by Steve Langasek to fix #716948 (an issue with dpkg-maintscript-helper rm_conffile messing with conffiles that the package no longer owns). Git mail notification. When I was still administrator of Alioth, I wrote git-commit-notice (a fork of Git s post-receive-email) and many packaging projects are using this hook script to send commit notices to mailing lists. This script has not been updated for multiple years and it started spewing warnings recently due to deprecated features in Wheezy s git. So I looked at updating it and while doing so I discovered a much better replacement with git-multimail. Thus I adapted git-commit-notice to work on top of this new script. The result has now been installed on git.debian.org (this is to be properly announced in the next DeveloperNews). Misc work. I packaged sql-ledger 3.0.5-1, forwarded #714739 on publican, and I participated in discussions to move the French Debian planets to planet.debian.org. Thanks See you next month for a new summary of my activities.

10 comments Liked this article? Click here. My blog is Flattr-enabled.

8 May 2013

Steve Langasek: Plymouth is not a bootsplash

Congrats to the Debian release team on the new release of Debian 7.0 (wheezy)! Leading up to the release, a meme making the rounds on Planet Debian has been to play a #newinwheezy game, calling out some of the many new packages in 7.0 that may be interesting to users. While upstart as a package is nothing new in wheezy, the jump to upstart 1.6.1 from 0.6.6 is quite a substantial change. It does bring with it a new package, mountall, which by itself isn't terribly interesting because it just provides an upstart-ish replacement for some core scripts from the initscripts package (essentially, /etc/rcS.d/*mount*). Where things get interesting (and, typically, controversial) is the way in which mountall leverages plymouth to achieve this. What is plymouth? There is a great deal of misunderstanding around plymouth, a fact I was reminded of again while working to get a modern version of upstart into wheezy. When Ubuntu first started requiring plymouth as an essential component of the boot infrastructure, there was a lot of outrage from users, particularly from Ubuntu Server users, who believed this was an attempt to force pretty splash screen graphics down their throats. Nothing could be further from the truth. Plymouth provides a splash screen, but that's not what plymouth is. What plymouth is, is a boot-time I/O multiplexer. And why, you ask, would upstart - or mountall, whose job is just to get the filesystem mounted at boot - need a boot-time I/O multiplexer? Why use plymouth? The simple answer is that, like everything else in a truly event-driven boot system, filesystem mounting is handled in parallel - with no defined order. If a filesystem is missing or fails an fsck, mountall may need to interact with the user to decide how to handle it. And if there's more than one missing or broken filesystem, and these are all being found in parallel, there needs to be a way to associate each answer from the user to the corresponding question from mountall, to avoid crossed signals... and lost data. One possible way to handle this would be for mountall to serialize the fsck's / mounts. But this is a pretty unsatisfactory answer; all other things (that is, boot reliability) being equal, admins would prefer their systems to boot as fast as possible, so that they can get back to being useful to users. So we reject the idea of solving the problem of serializing prompts by making mountall serialize all its filesystem checks. Another option would be to have mountall prompt directly on the console, doing its own serialization of the prompts (even though successful mounts / fscks continue to be run in parallel). This, too, is not desirable in the general case, both because some users actually would like to have pretty splash screens at boot time, and this would be incompatible with direct console prompting; and because mountall is not the only piece of software that need to prompt at boot time (see also: cryptsetup). Plymouth: not just a pretty face Enter plymouth, which provides the framework for serializing requests to the user while booting. It can provide a graphical boot splash, yes; ironically, even its own homepage suggests that this is its purpose. But it can also provide a text-only console interface, which is what you get automatically when booting without a splash boot argument, or even handle I/O over a serial console. Which is why, contrary to the initial intuitions of the s390 porters upon seeing this package, plymouth is available for all of Debian's Linux architectures in wheezy, s390 and s390x included, providing a consistent architecture for boot-time I/O for systems that need it - which is any machine using a modern boot system, such as upstart or systemd. Room for improvement Now, having a coherent architecture for your boot I/O is one thing; having a bug-free splash screen is another. The experience of plymouth in Ubuntu has certainly not been bug-free, with plymouth making significant demands of the kernel video layer. Recently, the binary video driver packages in Ubuntu have started to blacklist the framebuffer kernel driver entirely due to stability concerns, making plymouth splash screens a non-starter for users of these drivers and regressing the boot experience. One solution for this would be to have plymouth offload the video handling complexity to something more reliable and better tested. Plymouth does already have an X backend, but we don't use that in Ubuntu because even if we do have an X server, it normally starts much later than when we would want to display the splash screen. With Mir on the horizon for Ubuntu, however, and its clean separation between system and session compositors, it's possible that using a Mir backend - that can continue running even after the greeter has started, unlike the current situation where plymouth has to cede the console to the display manager when it starts - will become an appealing option. This, too, is not without its downsides. Needing to load plymouth when using crypted root filesystems already makes for a bloated initramfs; adding a system compositor to the initramfs won't make it any better, and introduces further questions about how to hand off between initramfs and root fs. Keeping your system compositor running from the initramfs post-boot isn't really ideal, particularly for low-memory systems; whereas killing the system compositor and restarting it will make it harder to provide a flicker-free experience. But for all that, it does have its architectural appeal, as it lets us use plymouth as long as we need to after boot. As the concept of static runlevels becomes increasingly obsolete in the face of dynamic systems, we need to design for the world where the distinction between "booting" and "booted" doesn't mean what it once did.

26 November 2012

Steve Langasek: Upstart in Debian

Good news, everyone! So as of last Sunday, this works on all Linux archs in Debian unstable and gives you a modern version of upstart:
echo 'Yes, do as I say!'   apt-get -o DPkg::options=--force-remove-essential -y --force-yes install upstart
Thanks to the ifupdown, sysvinit, and udev maintainers for their cooperation in getting upstart support in place; to the Debian release team for accomodating the late changes needed for upstart to be supported in wheezy; and to Scott for his past maintenance of upstart in Debian. Benchmarking One of the consequences is that it's now possible to do meaningful head-to-head comparisons of boot speed between sysvinit (with startpar), upstart, and systemd. At one time or another people have tested systemd vs. sysvinit when using bash as /bin/sh, and upstart vs. sysvinit, and systemd vs. sysvinit+startpar, and there are plenty of bootcharts floating around showing results of one init system or another on one distro or another, but I'm not aware of anyone having done a real, fair comparison of the three solutions, changing nothing but the init system. I've done some initial comparisons in a barebones sid VM, and the results are definitely interesting. Sysvinit with startpar (the default in Debian) can boot a stock sid install, with no added services, in somewhere between 3.37 and 3.42 seconds (three runs). That's not a whole lot, but on the other hand this is a system with a single filesystem and no interesting services yet. Is this really as fast as we can boot? No, even this minimal system can boot faster. Testing with upstart shows that upstart can do the same job in between 3.03 and 3.19 seconds (n=3, mean=3.09). This confirms what we'd already seen in Ubuntu, that it makes a difference to boot speed to have filesystem mounting handled by an integrated process that understands the whole system instead of as a group of serialized shell scripts. What about systemd? The same test gives a boot time between 2.32 and 2.85 seconds (n=4, mean=2.48). Interesting; what would make systemd faster than upstart in sid? Well, a quick look at the system shows one possible contributing factor: the rsyslog package in Debian has a systemd unit file, but not an upstart job file. Dropping in the /etc/init/rsyslog.conf from Ubuntu has a noticeable impact, and brings the upstart boot time down nearer to that of systemd (2.78-3.03s, n=5, mean=2.92). Besides telling me that it's time to start spamming Debian maintainers with wishlist bugs asking them to include upstart jobs in their packages, this suggests that the remaining difference in boot time may be due to the outstanding init scripts in rcS.d that are made built-in no-ops by systemd but not (yet) by upstart in Debian (e.g., hwclock, hostname, udev-mtab). (In Ubuntu, /etc/rcS.d has long since been emptied out in favor of upstart jobs in the common case, since the time it takes to get to runlevel=2 is definitely a major issue for boot speed and boot parallelization.) It also gives the lie to the claim that's been made in various places that spawning shells is a major bottleneck for upstart vs. systemd. More study is certainly needed to confirm this, but at least this naive first test suggests that in spite of the purported benefits of hard-coding boot-time policies in C code, upstart with its default degree of runtime configurability is at least in the ballpark of systemd. Indeed, when OpenSuSE switched from upstart to systemd, it seems that something else in the stack managed to nullify any benefit from improving the boot-time performance of apparmor. Contrary to what some would have you believe, systemd is not some kind of silver bullet for boot speed. Upstart, with its boot-time flexibility and its long history of real-world testing in Ubuntu, is a formidable competitor to systemd in the boot speed department - and a solid solution to the many longstanding boot-time ordering bugs in Debian, which still affect users of sysvinit. I've published the bootcharts for the above tests here. Between the fact that Debian's bootchart package logs by default to /var/log/bootchart.tgz (thus overwriting on every boot) and the fact that these tests are in a VM, I haven't bothered to include the raw data, just the bootcharts themselves. The interested reader can probably generate more interesting boot charts of their own anyway - in particular, it will be interesting to see how the different init systems perform with more complicated filesystem layouts, or when booting a less trivial set of services. Other musings The boot charts have been created with the bootchart package rather than with bootchart2. For one thing, it turns out that bootchart2 includes systemd units, not init scripts; so when replacing bootchart with bootchart2, the non-matching init script is left behind and systemd in particular gets terribly confused. This is now reported as Bug #694403. In an amusing twist, while I was experimenting with bootchart2, I also noticed that having systemd installed would slow down booting with other init systems, because systemd installs udev rules which take a noticeable amount of time to run a helper command at boot even though the helper should be a no-op. So if you're doing boot speed testing of other init systems, be sure you don't have systemd on the system at the time!

19 November 2012

Steve Langasek: Pflaumenkuchen

This was a good year for plums in the garden, both for the yellow plums and for the Italian prunes - enough so that it took some doing to figure out what to do with them all. Since I'm not in a hurry to set up a still and make slivovic, and you can only pawn so many plums off on friends and neighbors, I had on the order of 15 pounds of Italian prunes to dispense with. With our change of diet to eliminate extra carbs, Patty and I have both been experimenting with reduced-carb desserts in the kitchen. And I've always been fond of central European (e.g., German) desserts, which tend to be sweetened much more lightly than American equivalents. Indeed, my earliest impression of "coffee cake" comes from the home of an elderly couple who were friends of the family, who served a delicious plum cake in their home. She was from Bavaria, so I guess she probably wouldn't have called it Zwetchgendatschi like the Austrians do (if they even really call it that; I have my suspicions that this is a made-up name designed to poke fun at the foreigners). Maybe she would have called it Pflaumenkuchen. The Internet calls it "Polish plum cake". Whatever you call it, it's tasty, and I aimed to recreate it at home. The plums are now nearly a month gone, but after multiple iterations the cake turned out well enough that I feel compelled to share it with the Internet. Since we have friends who are dairy-intolerant, this particular recipe is tailored to be low-carb and dairy-free. It doesn't lose anything in the taste for being made with a restricted set of ingredients. The original recipe did call for a streusel, and a non-dairy streusel is pretty pointless so this recipe simply omits it. Metric cooks, avert your eyes. ;)
  • 1 c almond flour
  • 1 tsp baking powder
  • tsp salt
  • c Steviva blend
  • 3 tbsp grapeseed oil
  • c milk
  • 1 large egg
  • 12 fresh plums, pitted and halved
  • tsp cloves
Lightly coat an 8"x8" pan with cooking spray. Heat oven to 350 degrees. In medium bowl, combine flour, baking powder, salt and Steviva. Add grapeseed oil, milk and egg. Beat at medium speed 4 minutes. Pour batter into prepared pan. Place plums on top, cut side up, pushing down slightly into batter. Sprinkle cloves over the plums. Bake about 50 minutes or until a toothpick tests clean. Let the cake cool in the pan so the plum juices will be reabsorbed to create a moist cake. Sprinkle with confectioners' sugar, if desired, and cut into 16 squares.
[edit: the Internet is bad at me for my badly spelled german. Spelling corrected. ;)]

Steve Langasek: Pflaumkuchen

This was a good year for plums in the garden, both for the yellow plums and for the Italian prunes - enough so that it took some doing to figure out what to do with them all. Since I'm not in a hurry to set up a still and make slivovic, and you can only pawn so many plums off on friends and neighbors, I had on the order of 15 pounds of Italian prunes to dispense with. With our change of diet to eliminate extra carbs, Patty and I have both been experimenting with reduced-carb desserts in the kitchen. And I've always been fond of central European (e.g., German) desserts, which tend to be sweetened much more lightly than American equivalents. Indeed, my earliest impression of "coffee cake" comes from the home of an elderly couple who were friends of the family, who served a delicious plum cake in their home. She was from Bavaria, so I guess she probably wouldn't have called it Zwetchgendatschi like the Austrians do (if they even really call it that; I have my suspicions that this is a made-up name designed to poke fun at the foreigners). Maybe she would have called it Pflaumkuchen. The Internet calls it "Polish plum cake". Whatever you call it, it's tasty, and I aimed to recreate it at home. The plums are now nearly a month gone, but after multiple iterations the cake turned out well enough that I feel compelled to share it with the Internet. Since we have friends who are dairy-intolerant, this particular recipe is tailored to be low-carb and dairy-free. It doesn't lose anything in the taste for being made with a restricted set of ingredients. The original recipe did call for a streusel, and a non-dairy streusel is pretty pointless so this recipe simply omits it. Metric cooks, avert your eyes. ;)
  • 1 c almond flour
  • 1 tsp baking powder
  • tsp salt
  • c Steviva blend
  • 3 tbsp grapeseed oil
  • c milk
  • 1 large egg
  • 12 fresh plums, pitted and halved
  • tsp cloves
Lightly coat an 8"x8" pan with cooking spray. Heat oven to 350 degrees. In medium bowl, combine flour, baking powder, salt and Steviva. Add grapeseed oil, milk and egg. Beat at medium speed 4 minutes. Pour batter into prepared pan. Place plums on top, cut side up, pushing down slightly into batter. Sprinkle cloves over the plums. Bake about 50 minutes or until a toothpick tests clean. Let the cake cool in the pan so the plum juices will be reabsorbed to create a moist cake. Sprinkle with confectioners' sugar, if desired, and cut into 16 squares.

28 October 2012

Steve Langasek: SecureBoot in Ubuntu 12.10

The 12.10 release is the first version of Ubuntu that supports Secure Boot out of the box. In what is largely an accident of release timing, from what I can tell (and please correct me if I'm wrong), this actually makes Ubuntu 12.10 the first general release of any OS to support Secure Boot. (Windows 8 of course is also now available; and I'm sure Matthew Garrett, who has been a welcome collaborator throughout this process, has everything in good order for the upcoming Fedora 18 release.) That's certainly something of a bittersweet achievement. I'm proud of the work we've done to ensure Ubuntu will continue to work out of the box on the consumer hardware of the future; in spite of the predictable accusations on the blogowebs that we've sold out, I sleep well at night knowing that this was the pragmatic decision to make, maximizing users' freedom to use their hardware. All the same, I worry about what the landscape is going to look like in a few years' time. The Ubuntu first-stage EFI bootloader is signed by Microsoft, but the key that is used for signing is one that's recommended by Microsoft, not one that's required by the Windows 8 certification requirements. Will all hardware include this key in practice? The Windows 8 requirements also say that every machine must allow the user to disable Secure Boot. Will manufacturers get this right, and will users be able to make use of it in the event the manufacture didn't include the Microsoft-recommended key? Only time will tell. But I do think the Linux community is going to have to remain engaged on this for some time to come, and possibly hold OEMs' feet to the fire for shipping hardware that will only work with Windows 8. But that's for the future. For now, we have a technical solution in 12.10 that solves the parts of the problem that we can solve. This first release gives us preliminary support for booting on Secure Boot, but there's more work to be done to provide a full solution that's sustainable over the long term. We'll be discussing some of that this week at UDS in Copenhagen. And as part of our committment to enabling new hardware on the current LTS release, we will be backporting this work for inclusion in 12.04.2. It remains to be decided how Debian will approach the Secure Boot question. At DebConf 12, many people seemed to consider it a foregone conclusion that Debian would never agree to include binaries in main signed by third-party keys. I don't think that should be a given; I think allowing third-party signatures in main for hardware compatibility is consistent with Debian's principles, and refusing to make Debian compatible with this hardware out of the box does nothing to advance user freedom. I hope to see frank discussion post-wheezy about keeping Debian relevant on consumer hardware of the future.

15 July 2012

Steve Langasek: For those who care about my foot

Sitting in the airport during a layover on my way back from DebConf is one of the few times of year that time and motivation come together and routinely lead me to blog. Each year's DebConf has been special in its own way. This year at DebConf 12, in addition to the usual routine of catching up with all my far-away Debian friends, and of making new Debian friends in a country I've never been before, I got a special treat: my first opportunity to experience the practice of medicine in a foreign country. Last Saturday after arriving in Managua, I discovered that the very pretty paving stones making up the hotel walkway were also very slippery when wet, and the thin rubber strips that had been helpfully put in place to make them less slippery did not measurably succeed in doing so. So as it was raining that afternoon (as it does nearly every afternoon this time of year in Nicaragua), I slipped, and made the critical error of not falling. Instead, I stopped my slide - as it happens, by using the big toe of my left foot as if it were a roller skate brake. I don't recommend this, particularly if, as in my case, you already had it on your todo list to consult a podiatrist after DebConf regarding an ongoing feeling of weakness in precisely that part of precisely that foot. So while everything seemed fine the next day, Monday I woke up early with excruciating pain in my foot which, having forgotten about the incident two days prior, seemed to come from nowhere. After favoring the foot for another day and trying to keep the pain at bay using just OTC drugs, by Tuesday my foot was quite swollen and a source of concern. So I visited the clinic on the university campus, where the doctor diagnosed me with a dropped arch plus cellulitis due to bacteria entering through some invisible cut. Conclusion: university campus clinics are equally useless the world over. But at least in addition to pointlessly prescribing antibiotics, the doctor also prescribed some anti-inflammatory drugs to help bring the swelling down. Nevertheless, my questionable mobility kept both me and Patty from attending the day trip on Wednesday, which was a real shame as I had been keen to climb a volcano that day with my fellow Debian Developers (which in the end didn't happen for anyone, but that's a story for someone else's blog). Instead I took it easy for the day at the hotel, together with a few others who had opted not to attend the day trip, catching up on email and other work and hoping matters would improve. But by the end of the day the swelling was as bad as ever, if not worse, and I had no idea what was going on. Was it broken? Was something dislocated? And so I decided that the following day I would go to the hospital. The nearest hospital, as given in the DebConf local information page, is the Hospital Militar. It's a strange sort of thing as an American who remembers the 1980s to be seeking medical care from a facility operated by the Nicaraguan military; but accompanied by Fitoria, my native escort from the local team I had nothing to worry about; and besides, as a frequent international traveler I long ago learned the secret truth that people are people everywhere, no matter what flag they were born under. The highlight of the hospital visit was definitely when a member of the hospital staff, in full military uniform, shooed me towards a chair instead of standing to talk to her because "every patient has the right to sit, no matter what country they're in". After six x-rays and a doctor's consult, the conclusion was that it's probably not a broken bone and that it will probably clear up in just a couple of days with an even-better anti-inflammatory prescription and taking care of it by not walking on it. And indeed, my foot was already feeling better by Friday and aside from a little tenderness is now just about 100%. There are plenty of other entries on Planet Debian talking about the conference itself, and I don't think I have much to add to what's already been said there. But as for Nicaragua, I can conclude that it must be one of the best countries in the world to break your foot in - between the help of the hotel staff, the top-notch medical professionals, and above all the support of the DebConf local team (with a very big thank you again to Fitoria!), this is one brand of socialized medicine I can appreciate.

19 April 2012

Raphaël Hertzog: People behind Debian: Samuel Thibault, working on accessibility and the Hurd

Samuel Thibault is a French guy like me, but it took years until we met. He tends to keep a low profile, even though he s doing lots of good work that deserves to be mentioned. He focuses on improving Debian s accessibility and contributes to the Hurd. Who said he s a dreamer? :-) Checkout his interview to have some news of Wheezy s status on those topics. Raphael: Who are you? Samuel: I am 30 years old, and live in Bordeaux, France. During the workday, I teach Computer Science (Architecture, Networking, Operating Systems, and Parallel Programming, roughly) at the University of Bordeaux, and conduct researches in heterogeneous parallel computing. During the evening, I play the drums and the trombone in various orchestra (harmonic/symphonic/banda/brass). During the night, I hack on whatever fun things I can find, mainly accessibility and the Hurd at the moment, but also miscellaneous bits such as the Linux console support. I am also involved in the development of Aquilenet, an associative ISP around Bordeaux, and getting involved in the development of the network infrastructure in Bordeaux. I am not practicing Judo any more, but I roller-skate to work, and I like hiking in the mountains. I also read quite a few mangas. Saturday mornings do not exist in my schedule (Sunday mornings do, it s Brass Band rehearsal :) ). Raphael: How did you start contributing to Debian? Samuel: Bit by bit. I have been hacking around GNU/Linux since around 1998. I installed my first Debian system around 2000, as a replacement for my old Mandrake installation (which after all my tinkering was actually no longer looking like a Mandrake system any more!). That was Potato at the time, which somebody offered me through a set of CDs (downloading packages over the Internet was unthinkable at the time with the old modems). I have been happily reading and hacking around documentation, source code, etc. provided on them. Contribution things really started to take off when I went to the ENS Lyon high school in 2001: broadband Internet access in one s own student room! Since sending a mail was then really free, I started submitting bugs against various packages I was using. Right after that I started submitting patches along them, and then patches to other bugs. I did that for a long time actually. I had very little knowledge of all packaging details at the time, I was just a happy hacker submitting reports and patches against the upstream source code. At ENS Lyon, I met a blind colleague with very similar hacking tastes (of course we got friends) and he proposed me, for our student project, to work on a brlnet project (now called brlapi), a client/server protocol that lets applications render text on braille devices themselves. Along the way, I got to learn in details how a blind person can use a Unix system and the principles that should be followed when developing Accessibility. That is how I got involved in it. We presented our project at JDLL, and the Hurd booth happened to be next to our table, so I discussed with the Hurd people there about how the Hurd console could be used through braille. That is how I got into the Hurd too. From then on, I progressively contributed more and more to the upstream parts of both accessibility software and the Hurd. And then to the packaging part of them. Through patches in bug reports first, as usual, as well as through discussions on the mailing lists. But quickly enough people gave me commit access so I could just throw the code in. I was also given control over the Hurd buildds to keep them running. It was all good at that stage: I could contribute in all the parts I was caring about. People however started telling me that I should just apply for being a Debian Developer; both from accessibility and Hurd sides. I had also seen a bunch of my friends going through the process. I was however a bit scared (or probably it was just an excuse) by having to manage a gpg key, it seemed like a quite dangerous tool to me (even if I already had commit access to glibc at the time anyway ). I eventually applied for DM in 2008 so as to at least be able to upload some packages to help the little manpower of the Accessibility and Hurd teams. Henceforth I had already a gpg key, thus no excuse any more. And having it in the DM keyring was not enough for e.g. signing the hurd-i386 buildd packages. So I ended up going through NM in 2009, which went very fast, since I had already been contributing to Debian and learning all the needed stuff for almost 10 years! I now have around 50 packages in my QA page, and being a DD is actually useful for my work, to easily push our software to the masses :) So to sum it up, the Debian project is very easy to contribute to and open to new people. It was used during discussions at the GNU Hackers Meeting 2011 as an example of a very open community with public mailing lists and discussions. The mere fact that anybody can take the initiative of manipulating the BTS (if not scared by the commands) without having to ask anybody is an excellent thing to welcome contributions; it is notable tha the GNU project migrated to the Debbugs BTS. More generally, I don t really see the DD status as a must, especially now that we have the DM status (which is still a very good way to drag people into becoming DDs). For instance, I gave a talk at FOSDEM 2008 about the state of accessibility in Debian. People did not care whom I was, they cared that there was important stuff going on and somebody talking about it. More generally, decisions that are made through a vote are actually very rare. Most of the time, things just happen on the mailing lists or IRC channels where anybody can join the discussion. So I would recommend beginners to first use the software, then start reporting bugs, then start digging in the software to try fix the bugs by oneself, eventually propose patches, get them reviewed. At some point the submitted patches will be correct already most of the time. That s when the maintainers will start getting bored of just applying the patches, and simply provide with commit access, and voil , one has become a main contributor. Raphael: You re one of the main contributors to the Debian GNU/Hurd port. What motivates you in this project? Samuel: As I mentioned above, I first got real contact with the Hurd from the accessibility point of view. That initially brought me into the Hurd console, which uses a flexible design and nice interfaces to interact with it. The Hurd driver for console accessibility is actually very straightforward, way simpler than the Windows or Linux drivers. That is what caught me initially. I have continued working on it for several reasons. First, the design is really interesting for users. There are many things that are natural in the Hurd while Linux is still struggling to achieve them, such as UID isolation, recently mentioned in LWN. What I really like in the Hurd is that it excels at providing users with the same features as the administrator s. For instance, I find it annoying that I still can not mount an ISO image that I build on e.g. ries.debian.org. Linux now has FUSE which is supposed to permit that, but I have never seen it enabled on an ssh-accessible machine, only on desktop machines, and usually just because the administrator happens to be the user of the machine (who could as well just have used sudo ) For me, it is actually Freedom #0 of Free Software: let the user run programs for any purpose, that is, combining things together all the possible ways, and not being prevented from doing some things just because the design does not permit to achieve them securely. I had the chance to give a Hurd talk to explain that at GHM 2011, whose main topic was extensibility , I called it GNU/Hurd AKA Extensibility from the Ground, because the design of the Hurd is basically meant for extensibility, and does not care whether it is done by root or a mere user. All the tools that root uses to build a GNU/Hurd system can be used by the user to build its own GNU/Hurd environment. That is guaranteed by the design itself: the libc asks for things not to the kernel, but to servers (called translators), which can be provided by root, or by the user. It is interesting to see that it is actually also tried with varying success in GNU/Linux, through gvfs or Plash. An example of things I love being able to do is: $ zgrep foo ~/ftp://cdn.debian.net/debian/dists/sid/main/Contents-*.gz On my Hurd box, the ~/ftp: directory is indeed actually served by an ftpfs translator, run under my user uid, which is thus completely harmless to the system. Secondly and not the least, the Hurd provides me with interesting yet not too hard challenges. LWN confirmed several times that the Linux kernel has become very difficult to significantly contribute to, so it is no real hacking fun any more. I have notably implemented TLS support in the Hurd and the Xen and 64bit support in the GNU Mach kernel used by the Hurd. All three were very interesting to do, but were already done for Linux (at least for all the architectures which I actually know a bit and own). It happens that both TLS and Xen hacking experience became actually useful later on: I implemented TLS in the threading library of our research team, and the Xen port was a quite interesting line on my CV for getting a postdoc position at XenSource :) Lastly, I would say that I am used to lost causes :) My work on accessibility is sometimes a real struggle, so the Hurd is almost a kind of relief. It is famous for his vapourware reputation anyway, and so it is fun to just try to contribute to it nevertheless. An interesting thing is that the opinion of people on the Hurd is often quite extreme, and only rarely neutral. Some will say it is pure vapourware, while others will say that it is the hope of humanity (yes we do see those coming to #hurd, and they are not always just trolls!). When I published a 0.401 version on 2011 April 1st, the comments of people were very diverse, and some even went as far as saying that it was horrible of us to make a joke about the promised software :) Raphael: The FTPmasters want to demote the Hurd port to the debian-ports.org archive if it doesn t manage a stable release with wheezy. We re now at 2 months of the freeze. How far are you from being releasable ? Samuel: Of course, I can not speak for the Debian Release team. The current progress is however encouraging. During Debconf11, Michael Banck and I discussed with a few Debian Release team members about the kind of goals that should be achieved, and we are near completion of that part. The Debian GNU/Hurd port can almost completely be installed from the official mirrors, using the standard Debian Installer. Some patches need some polishing, but others are just waiting for being uploaded Debian GNU/Hurd can start a graphical desktop and run office tools such as gnumeric, as well as the iceweasel graphical web browser, KDE applications thanks to Pino Toscano s care, and GNOME application thanks to Emilio Pozuelo Monfort s care. Of course, general textmode hacking with gcc/make/gdb/etc. just works smoothly. Thanks to recent work on ghc and ada by Svante Signell, the archive coverage has passed 76%. There was a concern about network board driver support: until recently, the GNU Mach kernel was indeed still using a glue layer to embed the Linux 2.2 or even 2.0 drivers (!). Finding a network board supported by such drivers had of course become a real challenge. Thanks to the GSoC work of Zheng Da, the DDE layer can now be used to embed Linux 2.6.32 drivers in userland translators, which was recently ACCEPTed into the archive, and thus brings way larger support for network boards. It also pushes yet more toward the Hurd design: network drivers as userland process rather than kernel modules. That said, the freeze itself is not the final deadline. Actually, freeze periods are rests for porters, because maintainers stop bringing newer upstream versions which of course break on peculiar architectures. That will probably be helpful to continue improving the archive coverage. Raphael: The kfreebsd port brought into light all the packages which were not portable between different kernels. Did that help the Hurd port or are the problems too different to expect any mutual benefit? Samuel: The two ports have clearly helped each other in many aspects. The hurd-i386 port is the only non-Linux one that has been kept working (at least basically) for the past decade. That helped to make sure that all tools (dpkg, apt, toolchain, etc.) were able to cope with non-Linux ports, and keep that odd-but-why-not goal around, and evidently-enough achievable. In return, the kFreeBSD port managed to show that it was actually releasable, at least as a technological preview, thus making an example. In the daily work, we have sometimes worked hand in hand. The recent porting efforts of the Debian Installer happened roughly at the same time. When fixing some piece of code for one, the switch-case would be left for the other. When some code could be reused by the other, a mail would be sent to advise doing so, etc. In the packaging effort, it also made a lot of difference that a non-Linux port is exposed as released architecture: people attempted by themselves to fix code that is Linuxish for no real reason. The presence of the kFreeBSD is however also sometimes a difficulty for the Hurd: in the discussions, it sometimes tends to become a target to be reached, even if the systems are not really comparable. I do not need to detail the long history of the FreeBSD kernel and the amount of people hacking on it, some of them full-time, while the Hurd has only a small handful of free-time hackers. The FreeBSD kernel stability has already seen long-term polishing, and a fair amount of the Debian software was actually already ported to the FreeBSD kernel, thanks to the big existing pure-FreeBSD hackerbase. These do not hold for the GNU/Hurd port, so the expectations should go along. Raphael: You re also very much involved in the Debian Accessibility team. What are the responsibilities of this team and what are you doing there? Samuel: As you would expect it, the Debian Accessibility team works on packaging accessibility-related packages, and helping users with them; I thus do both. But the goal is way beyond just that. Actual accessibility requires integration. Ideally enough, a blind user should be able to just come to a Debian desktop system, plug his braille device, or press a shortcut to enable speech synthesis, and just use the damn computer, without having to ask the administrator to install some oddly-named package and whatnot. Just like any sighted user would do. He should be able to diagnose why his system does not boot, and at worse be able to reinstall his computer all by himself (typically at 2am ). And that is hard to achieve, because it means discussing about integration by default of accessibility features. For instance, the Debian CD images now beep during at the boot menu. That is a precious feature that has been discussed between debian-boot and debian-accessibility for a few weeks before agreeing on how to do it without too much disturbance. Similarly, my proposition of installing the desktop accessibility engines has been discussed for some time before being commited. What was however surprisingly great is that when somebody brought the topic back for discussion, non-debian-accessibility people answered themselves. This is reassuring, because it means things can be done durably in Debian. On the installation side, our current status is that the stable Debian installer has a high contrast color theme, and several years ago, I have pushed toward making standard CD images automatically detect braille devices, which permits standalone installation. I have added to the Wheezy installer some software speech synthesis (which again brought discussion about size increase vs versatility etc.) for blind people who do not have a braille device. I find it interesting to work on such topic in Debian rather than another distribution, because Debian is an upstream for a lot of distributions. Hopefully they just inherit our accessibility work. It at least worked for the text installer of Ubuntu. Of course, the Accessibility team is looking for help, to maintain our current packages, but also introduce new packages from the TODO list or create some backports. One does not need to be an expert in accessibility: tools can usually be tested, at least basically, by anybody, without particular hardware (I do not own any, I contributed virtual ones to qemu). For new developments and ideas, it is strongly recommended to come and discuss on debian-accessibility, because it is easy to get on a wrong track that does not bring actual accessibility. We still have several goals to achieve: the closest one is to just fix the transition to gnome3, which has been quite bad for accessibility so far :/ On the longer run, we should ideally reach the scenario I have detailed above: desktop accessibility available and ready to be enabled easily by default. Raphael: What s the biggest problem of Debian? Samuel: Debian is famous for its heated debian-devel discussions. And some people eventually say this no fun any more . That is exemplified in a less extreme way in the debian-boot/accessibility discussions that I have mentioned above. Sometimes, one needs to have a real stubborn thick head to continue the discussion until finding a compromise that will be accepted for commit. That is a problem because people do not necessarily have so much patience, and will thus prefer to contribute to a project with easier acceptance. But it is also a quality: as I explained above, once it is there, it is apparently for good. The Ubuntu support of accessibility in its installer has been very diverse, in part due to quite changing codebase. The Debian Installer codebase is more in a convergence process. Its base will have almost not changed between squeeze and wheezy. That allowed the Debian Accessibility team to continue improving its accessibility support, and not have to re-do it. A wiki page explains how to test its accessibility features, and some non-debian-accessibility people do go through it. A problem I am much more frightened by is the manpower in some core teams. The Debian Installer, grub, glibc, Xorg, gcc, mozilla derivatives, When reading the changelogs of these, we essentially keep seeing the same very few names over and over. And when one core developer leaves, it is very often still the same names which appear again to do the work. It is hard to believe that there are a thousand DDs working on Debian. I fear that Debian does not manage to get people to work on core things. I often hear people saying that they do not even dare thinking about putting their hands inside Xorg, for instance. Xorg is complex, but it seems to me that it tends to be overrated, and a lot of people could actually help there, as well as all the teams mentioned above. And if nobody does it, who will? Raphael: Do you have wishes for Debian Wheezy? Samuel: That is an easy one :) Of course I wish that we manage to release the hurd-i386 port. I also wish that accessibility of gnome3 gets fixed enough to become usable again. The current state is worrying: so much has changed that the transition will be difficult for users already, the current bugs will clearly not help. I also hope to find the time to fix the qt-at-spi bridge, which should (at last!) bring complete KDE accessibility. Raphael: Is there someone in Debian that you admire for their contributions? Samuel: Given the concerns I expressed above, I admire all the people who do spend time on core packages, even when that is really not fun everyday. Just to alphabetically name a few people I have seen so often here and there in the areas I have touched in the last few years: Aur lien Jarno, Bastian Blank, Christian Perrier, Colin Watson, Cyril Brulebois, Frans Pop, J rg Jaspert, Joey Hess, Josselin Mouette, Julien Cristau, Matthias Klose, Mike Hommey, Otavio Salvador, Petr Salinger, Robert Millan, Steve Langasek. Man, so many things that each of them works on! Of course this list is biased towards the parts that I touched, but people working in others core areas also deserve the same admiration.
Thank you to Samuel for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

25 February 2012

Russ Allbery: Copyright format 1.0 published

Four and a half years after Sam Hocevar started a draft proposal on the Debian Wiki, we've published revision 1.0 of the machine-readable format for debian/copyright files. The format (as DEP-5) has been in widespread use for some time, but there have been multiple versions of the format and multiple transitional versioned URLs. Everyone can now update to the same version of the document and use a stable format URL. Below is the text of the annoucement that I just sent to debian-devel-announce. Version 1.0 of the machine-readable format for debian/copyright files (the culmination of the DEP-5 process) has now been published. The canonical URL (and also the URL to use in the Format field in such files) is:
http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
For those who have adopted various iterations of the DEP-5 format for your packages, please consider updating to this format the next time you revise your package. Use of this format in Debian packages is optional. Thanks to all of the people who have contributed to this work over its long evolution. Particular thanks go to Steve Langasek, Charles Plessy, and Lars Wirzenius, who kept this document moving through its extended DEP discussion process, and Sam Hocevar, who started this work in 2007. The copyright format specification is now being maintained as part of the debian-policy package and will follow the same update process that's used for other parts of Policy. See:
http://wiki.debian.org/PolicyChangesProcess
for that process. For this specification, we're trying something new for Debian technical policies. This document is versioned, and older versions of the specification will be maintained to not invalidate older references in copyright files that have not been moved to newer versions. This is similar to the way that specifications are handled by some other projects, particularly the freedesktop.org standards. The plan, going forward, is that informative changes (changes that do not change the requirements or the contents of compliant copyright files) will be made following the same relaxed procedure as other informative changes to Policy, and new versions will only informative changes will be published with the same version number. So the existing 1.0 document may receive further improvements in wording, examples, or similar changes that don't modify its meaning. Normative changes (changes that change requirements for compliant files, which includes addition of new standardized license short names) will follow the normative Policy change process, including formal seconds. Once normative changes are accepted, the next release of this specification will receive a new version number (1.1, for example). The previous version, prior to the normative changes, will be frozen, and from that point forward will not be changed except as required to continue to publish it. This includes further informative changes. We will continue publishing these frozen older versions of the standard on www.debian.org and in the debian-policy package for some time: possibly forever, and at least as long as they're still referred to by packages distributed by the project. The hope is that this will provide a good balance between specification stability, editing for clarity, and specification changes to meet changing requirements, while not invalidating the URLs in older files and while continuing to provide the information required to interpret older files.

1 February 2012

Cyril Brulebois: dpkg with multiarch

(This page was edited a few times, details are available.) Grab it while it s hot! Thanks to the hard work of dpkg developers and many (generations of) developers, multiarch is becoming a reality. If you want to give it a try, install dpkg from experimental, add a foreign architecture, and start trying to install packages. Example on amd64:
# dpkg --add-architecture i386
# dpkg --print-foreign-architectures
i386
# apt-get update
[ lots of amd64 and i386 Packages files get downloaded ]
# apt-get install mksh:i386
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  ed:i386
The following NEW packages will be installed:
  mksh:i386
0 upgraded, 1 newly installed, 0 to remove and 9 not upgraded.
Need to get 414 kB of archives.
After this operation, 707 kB of additional disk space will be used.
Get:1 http://ftp.fr.debian.org/debian/ sid/main mksh i386 40.4-2 [414 kB]
Fetched 414 kB in 0s (664 kB/s)
Selecting previously unselected package mksh.
(Reading database ... 171933 files and directories currently installed.)
Unpacking mksh (from .../archives/mksh_40.4-2_i386.deb) ...
Processing triggers for menu ...
Processing triggers for man-db ...
Setting up mksh (40.4-2) ...
update-alternatives: using /bin/mksh to provide /bin/ksh (ksh) in auto mode.
Processing triggers for menu ...
# mksh
$ ldd $(which mksh)
linux-gate.so.1 =>  (0xf779c000)
libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xf75d6000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf75b9000)
/lib/ld-linux.so.2 (0xf779d000)
Of course, installing an i386 mksh package isn t exactly what multiarch is about. Dozens of packages have been patched already to add Multi-Arch fields, but until their (recursive) dependencies have been multiarch-ified, foreign packages can be uninstallable, as can be seen below, with the usual why is it uninstallable? hunt (shortened output):
# apt-get install psutils:i386
The following packages have unmet dependencies:
 psutils:i386 : Depends: libpaper1:i386 but it is not going to be installed
# apt-get install libpaper1:i386
The following packages have unmet dependencies:
 libpaper1:i386 : Depends: ucf:i386 (>= 0.28) but it is not installable
                  Recommends: libpaper-utils:i386 but it is not going to be installed
# apt-get install ucf:i386
E: Package 'ucf:i386' has no installation candidate
Another example, successful handling of the installation of a foreign package, while it s already installed with the host architecture:
# apt-get install xz-utils:i386
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  liblzma5:i386
Suggested packages:
  xz-lzma:i386
The following packages will be REMOVED:
  xz-utils
The following NEW packages will be installed:
  liblzma5:i386 xz-utils:i386
0 upgraded, 2 newly installed, 1 to remove and 9 not upgraded.
Need to get 440 kB of archives.
After this operation, 410 kB of additional disk space will be used.
Do you want to continue [Y/n]?
Get:1 http://ftp.fr.debian.org/debian/ sid/main liblzma5 i386 5.1.1alpha+20110809-3 [205 kB]
Get:2 http://ftp.fr.debian.org/debian/ sid/main xz-utils i386 5.1.1alpha+20110809-3 [235 kB]
Fetched 440 kB in 0s (478 kB/s)
dpkg: xz-utils: dependency problems, but removing anyway as you requested:
 dpkg depends on xz-utils.
 xz-lzma depends on xz-utils.
 dpkg-dev depends on xz-utils.
(Reading database ... 171952 files and directories currently installed.)
Removing xz-utils ...
Processing triggers for man-db ...
Selecting previously unselected package liblzma5:i386.
(Reading database ... 171908 files and directories currently installed.)
Unpacking liblzma5:i386 (from .../liblzma5_5.1.1alpha+20110809-3_i386.deb) ...
Selecting previously unselected package xz-utils.
Unpacking xz-utils (from .../xz-utils_5.1.1alpha+20110809-3_i386.deb) ...
Processing triggers for man-db ...
Setting up liblzma5:i386 (5.1.1alpha+20110809-3) ...
Setting up xz-utils (5.1.1alpha+20110809-3) ...
# dpkg -l xz-utils xz-utils:i386 'liblzma*:*'
[   ]
un  xz-utils            <none>
ii  xz-utils            5.1.1alpha+20110809-3
ii  liblzma5:amd64      5.1.1alpha+20110809-3
ii  liblzma5:i386       5.1.1alpha+20110809-3
# ldd $(which xz)
    linux-gate.so.1 =>  (0xf776b000)
    liblzma.so.5 => /usr/lib/i386-linux-gnu/liblzma.so.5 (0xf7724000)
    librt.so.1 => /lib/i386-linux-gnu/i686/cmov/librt.so.1 (0xf771b000)
    libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0xf7701000)
    libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xf75a4000)
    /lib/ld-linux.so.2 (0xf776c000)
# zgrep -a '_("Activities")' gnome-shell-3.2.2.1.tar.xz
        this._label = new St.Label(  text: _("Activities")  );
zgrep is a shell script, but it calls xz, which is from the i386 package, and everything runs just fine. Running it through strace -f -e '', I discovered those messages, which I had never seen before:
[ Process PID=8900 runs in 32 bit mode. ]
[ Process PID=8899 runs in 64 bit mode. ]
[ Process PID=8900 runs in 32 bit mode. ]
[ Process PID=8899 runs in 64 bit mode. ]
 
What s next? We re late! It s time to check what happens with that dpkg package, report bugs, and convert more libraries! Please think of the kittens, and coordinate with the release team to make sure you don t delay a transition when uploading a shiny, multiarch-ified package. In a nutshell, if you received a patch from Steve Langasek or Riku Voipio, it s a good indication your package will be helpful quickly when it s multiarchified. Since zlib is directly depended on by 2000+ packages, #569697 was pinged right after the dpkg upload; but many other packages will need patches and heavy testing. Hurry up, the freeze is coming, we need to shake it up as soon as possible!

30 January 2012

Colin Watson: APT resolver bugs

I've managed to go for eleven years working on Debian and nearly eight on Ubuntu without ever needing to teach myself how APT's resolver works. I get the impression that there's a certain mystique about it in general (alternatively, I'm just the last person to figure this out). Recently, though, I had a couple of Ubuntu upgrade bugs to fix that turned out to be bugs in the resolver, and I thought it might be interesting to walk through the process of fixing them based on the Debug::pkgProblemResolver=true log files. Breakage with Breaks The first was Ubuntu bug #922485 (apt.log). To understand the log, you first need to know that APT makes up to ten passes of the resolver to attempt to fix broken dependencies by upgrading, removing, or holding back packages; if there are still broken packages after this point, it's generally because it's got itself stuck in some kind of loop, and it bails out rather than carrying on forever. The current pass number is shown in each "Investigating" log entry, so they start with "Investigating (0)" and carry on up to at most "Investigating (9)". Any packages that you see still being investigated on the tenth pass are probably something to do with whatever's going wrong. In this case, most packages have been resolved by the end of the fourth pass, but xserver-xorg-core is causing some trouble. (Not a particular surprise, as it's an important package with lots of relationships.) We can see that each breakage is:
Broken xserver-xorg-core:i386 Breaks on xserver-xorg-video-6 [ i386 ] < none > ( none )
This is a Breaks (a relatively new package relationship type introduced a few years ago as a sort of weaker form of Conflicts) on a virtual package, which means that in order to unpack xserver-xorg-core each package that provides xserver-xorg-video-6 must be deconfigured. Much like Conflicts, APT responds to this by upgrading providing packages to versions that don't provide the offending virtual package if it can, and otherwise removing them. We can see it doing just that in the log (some lines omitted):
Investigating (0) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-tseng:i386
Investigating (1) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-i740:i386
Investigating (2) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-nv:i386
OK, so that makes sense - presumably upgrading those packages didn't help at the time. But look at the pass numbers. Rather than just fixing all the packages that provide xserver-xorg-video-6 in a single pass, which it would be perfectly able to do, it only fixes one per pass. This means that if a package Breaks a virtual package which is provided by more than ten installed packages, the resolver will fail to handle that situation. On inspection of the code, this was being handled correctly for Conflicts by carrying on through the list of possible targets for the dependency relation in that case, but apparently when Breaks support was implemented in APT this case was overlooked. The fix is to carry on through the list of possible targets for any "negative" dependency relation, not just Conflicts, and I've filed a patch as Debian bug #657695. My cup overfloweth The second bug I looked at was Ubuntu bug #917173 (apt.log). Just as in the previous case, we can see the resolver "running out of time" by reaching the end of the tenth pass with some dependencies still broken. This one is a lot less obvious, though. The last few entries clearly indicate that the resolver is stuck in a loop:
Investigating (8) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (8) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
Investigating (9) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (9) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
The new version of dpkg requires upgrading dpkg-dev, but it can't because of something wrong with libdpkg-perl. Following the breadcrumb trail back through the log, we find:
Investigating (1) libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl )
Broken libdpkg-perl:i386 Depends on perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
  Considering perl:i386 1472 as a solution to libdpkg-perl:i386 12
  Holding Back libdpkg-perl:i386 rather than change perl:i386
Investigating (1) perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl:i386 Depends on perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl ) (= 5.14.2-6ubuntu1)
  Considering perl-base:i386 5806 as a solution to perl:i386 1472
  Removing perl:i386 rather than change perl-base:i386
Investigating (1) perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl-base:i386 PreDepends on libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (>= 2.11)
  Considering libc6:i386 -17473 as a solution to perl-base:i386 5806
  Added libc6:i386 to the remove list
Investigating (0) libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs )
Broken libc6:i386 Depends on libc-bin [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (= 2.11.1-0ubuntu7.8)
  Considering libc-bin:i386 10358 as a solution to libc6:i386 -17473
  Removing libc6:i386 rather than change libc-bin:i386
So ultimately the problem is something to do with libc6; but what? As Steve Langasek said in the bug, libc6's dependencies have been very carefully structured, and surely we would have seen some hint of it elsewhere if they were wrong. At this point ideally I wanted to break out GDB or at the very least experiment a bit with apt-get, but due to some tedious local problems I hadn't been able to restore the apt-clone state file for this bug onto my system so that I could attack it directly. So I fell back on the last refuge of the frustrated debugger and sat and thought about it for a bit. Eventually I noticed something. The numbers after the package names in the third line of each of these log entries are "scores": roughly, the more important a package is, the higher its score should be. The function that calculates these is pkgProblemResolver::MakeScores() in apt-pkg/algorithms.cc. Reading this, I noticed that the various values added up to make each score are almost all provably positive, for example:
         Scores[I->ID] += abs(OldScores[D.ParentPkg()->ID]);
The only exceptions are an initial -1 or -2 points for Priority: optional or Priority: extra packages respectively, or some values that could theoretically be configured to be negative but weren't in this case. OK. So how come libc6 has such a huge negative score of -17473, when one would normally expect it to be an extremely powerful package with a large positive score? Oh. This is computer programming, not mathematics ... and each score is stored in a signed short, so in a sufficiently large upgrade all those bonus points add up to something larger than 32767 and everything goes haywire. Bingo. Make it an int instead - the number of installed packages is going to be on the order of tens of thousands at most, so it's not as though it'll make a substantial difference to the amount of memory used - and chances are everything will be fine. I've filed a patch as Debian bug #657732. I'd expected this to be a pretty challenging pair of bugs. While I certainly haven't lost any respect for the APT maintainers for dealing with this stuff regularly, it wasn't as bad as I thought. I'd expected to have to figure out how to retune some slightly out-of-balance heuristics and not really know whether I'd broken anything else in the process; but in the end both patches were very straightforward.

29 January 2012

Gregor Herrmann: RC bugs 2012/04

good news: from looking through RC bugs in the BTS, it seems that more & more people are starting to join the RCBW effort!

& here's my usual list for the past week:

22 January 2012

Gregor Herrmann: RC bugs 2012/03

here's the list of RC bugs I've worked on during the last week:

1 January 2012

Gregor Herrmann: RC bugs 2011/52

a rather lazy bugsquashing week but with a little cheating a managed to get at at least 7 RC bugs:

6 December 2011

Steve Langasek: Making jam from bugs

This weekend, we held a combined Debian Bug Squashing Party and Ubuntu Local Jam in Portland, OR. A big thank you to PuppetLabs for hosting! Thanks to a brilliant insight from Kees Cook, we were able to give everyone access to their own pre-configured build environment as soon as they walked in the door by deploying schroot/sbuild instances in "the cloud" (in this case, Amazon EC2). Small blips with the mirrors notwithstanding, this worked out pretty well, and let people start to get their hands dirty as soon as they walked in the door instead of spending a lot of time up front doing the boring work of setting up a build environment. This was a big win for people who had never done a package build before, and I highly recommend it for future BSPs. You can read about the build environment setup in the Debian wiki, and details on setting up your own BSP cloud in Kees's blog. (And the cloud instances were running Ubuntu 11.10 guests, with Debian unstable chroots - a perfect pairing for our joint Debian/Ubuntu event!) So how did this curious foray into a combined Ubuntu/Debian event go? Not too shabby: When all was said and done, we didn't get a chance to tackle any wheezy release critical bugs like we'd hoped. That's ok, that leaves us something to do for our next event, which will be bigger and even better than this one. Maybe even big enough to rival one of those crazy, all-weekend BSPs that they have in Germany...

Next.

Previous.